Correction to M-test with resampling, and addition of new MLL test #268
Conversation
…ampling from the entire stochastic set is possible now with the full_calculation flag. Removed mag_half_bin, which is now calculated directly from the region magnitude bins.
…nitude_test and MLL_score. docs: updated URLs for intersphinx. Added new M-tests to the API reference.
There was a problem hiding this comment.
Functions are looking great!!
Unit tests were created for simple catalog/forecasts. The function results were reproduced by hand calculation, or my third-person implementation of the scores from just reading the manuscript.
California Landers forecast and catalog (pycsep) were tested with the function, and provided same result as given by author. Didn't placed this in the unit-testing suite, since its too costly. Should be placed into QA tests with the other tests in the future.
Only pendings are to write a docstring of log_d_multinomial (@Serra314), confirm that it is a +2 and not -2 in the return value of the MLL_score (@Serra314) and write a documentation snippet in the documentation: (1) Getting Started / Theory of CSEP Tests and (2) Concepts /Evaluations.
|
|
||
|
|
||
| def log_d_multinomial(x: numpy.ndarray, size: int, prob: numpy.ndarray): | ||
| """ |
There was a problem hiding this comment.
This requires docstrings explaining how it was calculated and/or derived from the MS.
There was a problem hiding this comment.
It's the multinomial log-likelihood which is given by
log( Γ(size + 1) ) + Σ_i x_i log(prob_i ) - log( Γ(x_i + 1) )
| N_j = numpy.sum(catalog_counts) | ||
| events_ratio = N_u / N_j | ||
|
|
||
| union_catalog_counts_mod = union_catalog_counts + events_ratio |
There was a problem hiding this comment.
I changed the named Lambda_U and Lambda_j to this more python variable names (No start with caps). Let me know if u agree :D or have other suggestion.
| size=numpy.sum(catalog_counts_mod), | ||
| prob=pr_cat_j) | ||
|
|
||
| return 2 * (log_lik_merged - log_lik_union - log_like_cat_j) |
There was a problem hiding this comment.
Shouldn't this be a negative -2?? Or did we flip the score? I'm always confused when flipping.
There was a problem hiding this comment.
Yes it should according to the definition in the manuscript. I'll change that. The minus sign does not change any property of the MLL statistic and it just make it positively or negatively oriented, so no issue changing the definition.
…ft Serafini et al., 2024 as in-prep, until pre-print is submitted.
pyCSEP Pull Request Checklist
Please check out the contributing guidelines for some tips
on making pull requests to pyCSEP.
Fixes issue #(please fill in or delete if not needed).
Type of change:
Please delete options that are not relevant.
Checklist: